Private-Sector AI Indicators – Center for Security and Emerging Technology (CSET), Georgetown University https://eto.tech/dataset-docs/private-sector-ai-indicators/ Zenodo Record: https://zenodo.org/records/14518311

The source links directly to the official documentation page for the Private-Sector AI Indicators dataset, which is produced by the Center for Security and Emerging Technology (CSET) at Georgetown University. Because the dataset is published by a recognized academic research institution and distributed through Zenodo with a Digital Object Identifier (DOI), it is transparent and traceable. This indicates that the data comes from a credible research organization rather than an anonymous or synthetic source. The dataset reflects real-world corporate AI activity and is based on systematically collected information such as AI publications and patent filings. Since the documentation clearly explains how the dataset was constructed and what it measures, we can confirm that this is a real and reliable dataset suitable for analysis.

The dataset represents a collection of company-level indicators measuring AI activity in the private sector. Each row is a specific company, and the columns include quantitative measures such as AI-related research publications, AI-related patent filings, and workforce-related indicators. These variables capture different dimensions of AI innovation and technological engagement within companies. The information contained in this dataset is relevant to our project because our project aims to analyze patterns of AI activity across companies. This dataset provides measurable indicators that allow us to compare companies based on their AI research output and innovation intensity. By using this dataset, we can identify differences in AI development across firms and industries.

We plan to join the country-level aggregates from this dataset with our second dataset. Since the two datasets may use slightly different naming conventions for countries, we will standardize the country names before merging. We will convert all country names to a consistent format and ensure that spelling and capitalization match. We plan to do an inner join so that only countries appearing in both datasets are retained in the final merged dataset.

This dataset contains 692 observable rows with 64 columns measuring AI publication percentages and related company attributes. Each row represents a company, and the data includes variables that allow us to calculate average AI publication intensity at the country level. These columns are directly related to our project because they allow us to measure how strongly companies in different countries engage in AI research.

From the analyzed data, we observed that some companies may have missing country information or incomplete publication percentage values. Additionally, country names may appear in slightly different formats, such as USA vs United States. However, these issues can be resolved through standardization and aggregation. Overall, there are no major structural problems in the dataset that would prevent it from being used in our analysis.

Source Link: EU Industrial R&D Investment Scoreboard – World 2000 Edition European Commission, Joint Research Centre (JRC) https://iri.jrc.ec.europa.eu/scoreboard/2025-eu-industrial-rd-investment-scoreboard

The second dataset used in our project is the EU Industrial R&D Investment Scoreboard (World 2000). It is published by the European Commission through the Joint Research Centre (JRC) and the Directorate-General for Research and Innovation. This dataset reports R&D investment data for the world’s top 2000 corporate R&D investors.The data are compiled from audited company annual financial reports. This means the R&D expenditure values come directly from official company financial statements. The 2025 edition reflects company data from fiscal year 2024. Because the dataset is produced by a government research body and based on audited financial records, it qualifies as real and credible data. It measures actual corporate investment behavior rather than simulated or synthetic values.

This dataset contains company-level R&D investment data from different industries and countries. It focuses on overall research and development spending rather than AI-specific research.The dataset includes variables such as total R&D expenditure, R&D intensity, net sales, and number of employees. These variables show how much companies invest in innovation and how large they are.This information is relevant to our project because we can compare company R&D investment with AI research output from our first dataset. This helps us examine whether companies that spend more on R&D also produce more AI research.

We plan to join this dataset with our first dataset using company names. The key column in the Scoreboard is Company, and the key column in our first dataset is also the company name. Company names may not match exactly due to punctuation, abbreviations, or legal suffixes. We will standardize names by converting to lowercase, removing extra spaces, and removing punctuation. We will then perform an inner join to keep only companies that appear in both datasets.

This dataset contains 2000 rows and 24 columns. Each row represents one company among the top global R&D investors. The dataset includes financial and performance variables such as R&D spending, net sales, and employee counts.The R&D variable is measured in millions of euros. This allows us to directly compare the scale of company investment. The dataset also includes the country of headquarters, which can be used for additional aggregation if needed.

One issue in our combined dataset is the difference in measurement units between the two data sources. Dataset 1 measures AI output using patent counts and publication percentages. Dataset 2 measures total R&D investment in millions of euros and R&D intensity percentages. Because the units are different, we cannot directly compare raw values across the two datasets. AI patents represent output, while R&D spending represents input. This creates a scale and dimension difference. To address this issue, we created a standardized efficiency variable. We calculated AI patents per €100 million of R&D spending. This allows us to compare companies more fairly and control for differences in company size and investment scale.

Our dataset may have several limitations. First, both datasets mainly include large companies. Small firms and startups are not fully represented. This creates selection bias. Second, the Scoreboard focuses on top global R&D investors. Companies from developing countries may be underrepresented. Third, AI publication data only measure published research. Companies may conduct private AI research that is not included.

The datasets do not contain personal data. This reduces privacy risk. The data sources are publicly documented, which supports transparency and accountability. However, the data do not measure social impact, worker outcomes, or fairness of AI systems. Therefore, the dataset does not fully reflect all human rights principles. Our analysis focuses only on corporate investment and research patterns. It should not be interpreted as a complete measure of ethical or societal impact of AI development.

• Duofei Cui (College of Arts & Sciences, University of Washington; duofec@uw.edu) is interested in analyzing patterns of technological innovation and understanding how corporate investment influences AI research output. She is particularly interested in applying data science methods to study complex systems and research trends across industries.
• Carol Zeng (College of Arts & Sciences, University of Washington; hxz63@uw.edu) is interested in patterns of technological innovation, game design, and human–centered interaction. She applies data‑science methods to analyze complex systems and track research trends across industries.

In this project, we used several tools beyond those directly covered in lecture.

• stringdist package (amatch function): We used fuzzy matching to join company names across datasets because the names were not identical. We learned how to use the amatch() function by reading the official R documentation and examples from CRAN. Resource: https://cran.r-project.org/web/packages/stringdist/stringdist.pdf.
• readxl package: We used read_excel() to import the EU Industrial R&D Investment Scoreboard dataset from an Excel file. This function was not heavily emphasized in class, so we referred to the official tidyverse documentation to understand its usage. Resource: https://readxl.tidyverse.org/
• Log scale transformations in ggplot2 : To better visualize skewed efficiency data, we used scale_y_log10() to adjust the axis scale. We reviewed examples from the ggplot2 documentation to implement this correctly. Resource: https://ggplot2.tidyverse.org/reference/scale_continuous.html
• All other data wrangling, filtering, grouping, and visualization steps were completed using tools covered in class, including dplyr and ggplot2.